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SUMMARY 


This  project  evaluated  the  validity  of  present  selection  measures  for  entry  Into  the  Basic 
Journalism  and  Basic  Broadcaster  courses  to  determine  If  the  English  Diagnostic  Test  (EOT),  given 
to  all  course  entrants,  adds  to  the  validity  of  the  present  test  composites  in  predicting 
training  outcomes.  Data  were  collected  on  228  enlisted  service  personnel  (from  the  Air  Force, 
Army.  Navy,  and  Marine  Corps)  who  enrolled  In  the  two  Journalism  courses  during  FY82.  Multiple 
regression  analyses  were  conducted  for  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB) 
composite  test  scores  and  the  EOT  raw  scores  using  final  school  grades  as  the  criterion.  Data 
were  also  analyzed  by  service  membership  to  determine  whether  the  results  of  the  full  sample  were 
applicable  for  specific  service  groups.  The  analyses  Indicated  that  the  Air  Force  General  test 
coaqiosite  (GEN),  which  Includes  Arithmetic  Reasoning,  Word  Knowledge,  and  Paragraph  Comprehension 
subtests,  had  higher  predictive  validity  than  the  EOT  or  any  of  the  other  service  composites.  A 
GEN  cutoff  score  at  the  6Sth  percentile  Is  recommended  for  entry  into  both  Journalism  courses. 


PREFACE 


This  Is  an  Interin  report  on  Inproving  the  selection  accuracy  of  students  1n  the 
Joint  service  training  program  In  Journal 1sn*re1ated  courses.  The  report  covers  the 
period  of  April  1983  to  November  1983.  It  was  completed  under  the  auspices  of  Personnel 
Qualifications,  which  Is  part  of  a  larger  effort  In  Force  Acquisition  and  Distribution. 
It  was  subsuned  under  project  nunber  77I9I8I9,  ■Oevetopnent  and  Validation  of  Selection 
Methodologies*  and  was  executed  as  part  of  the  Air  Force  Hunan  Resources  Laboratory's 
responsibility  for  Inproving  Air  Force  selection  and  classification  systens. 
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ENGLISH  DIAGNOSTIC  TEST:  VALIDATION  FOR 
JOURNAL ISK-RELATED  PROGRAMS 


I.  INTRODUCTION 

The  Defense  Inforaatlon  School  (DINFOS)  at  Fort  Benjamin  Harrison,  Indiana,  conducts 
training  for  the  Basic  Journalism  Course  (BJC)  and  the  Basic  Broadcaster  Course  (BBC).  Enlisted 
personnel  from  all  United  States  Armed  Services  attend  these  courses.  Each  service  has  specific 
aptitude  requirements  for  entry  Into  BJC  or  BBC.  These  requirements  are  minimum  composite  scores 
derived  from  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  (Ree,  Mullins,  Mathews.  & 
Massey.  1982),  which  is  administered  prior  to  enlistment.  The  composites  used  by  each  service 
and  the  minimum  aptitude  percentile  equivalent  scores  currently  needed  for  entry  Into  the  DINFOS 
courses  are  as  follows: 

Air  Force  •  General  (GEN)  70 

Narine  Corps  -  General-Technical  (GEN)  69 

Navy  -  Clerical  (CL)  50 

Army  -  Skilled  Technical  (ST)  SO 

The  subtests  that  compose  each  of  these  composites  are  shown  In  Table  t. 


Table  I.  Subtestt  and  Service  Composites 


Subtest 

GEN* 

Composite 

cP 

stc 

Arithmetic  Reasoning  (AR) 

X 

Word  Knowledge  (WK) 

X 

X 

X 

Paragraph  Comprehension  (PC) 

X 

X 

X 

Numerical  Operations  (NO) 

X 

Coding  Speed  (CS) 

X 

General  Science  (GS) 

X 

Mathematics  Knowledge  (NK) 

X 

Mechanical  Comprehension  (NC) 

X 

*G£N  •  Air  Force  and  Marine  Corps  General  composite. 
*’CL  •  Navy  Clerical  composite. 

‘^ST  •  Army  Skilled  Technical  composite. 


These  specific  classification  requirements  are  In  addition  to  minimum  ASVAB  test  scores 
required  for  service  qualification.  In  addition,  prospective  trainees  must  possess  basic  English 
language  skills.  For  several  years,  the  DINFOS  English  Diagnostic  Test  (EDT)  has  been 
administered  to  personnel  upon  entry  Into  BJC  or  BBC.  Those  scoring  below  40  on  this  64-lteffl 
test  are  given  3  weeks  of  remedial  training  before  beginning  regular  BJC  or  BBC  training.  The 
remedial  language  training  covers  grammar,  usage,  sentence  structure,  word  choice,  and 
capitalization. 

In  FY62,  535  service  personnel  entered  BJC.  Of  these,  16. It  were  academic  failures.  Of 
those  passing  the  EDT,  only  11.5%  were  BJC  academic  failures.  These  findings  suggested  that  the 
EDT  might  be  useful  In  screening  enlistees  prior  to  entry  Into  training.  Whether  this  test  had 
appreciable  predictive  validity  In  addition  to  that  possessed  by  current  ASVAB  measures,  however, 
had  not  been  subject  to  systematic  Investigation. 


The  purpose  of  the  present  study  was  twofold:  (a)  to  evaluate  the  validity  of  present 
selection  Measures  against  BJC  and  BBC  training  criteria  and  (b)  to  determine  If  the  EOT  adds  to 
the  validity  of  the  present  Measures  In  predicting  training  outcoMes. 


II.  METHOD 

Subjects 

The  data  were  based  on  228  enlisted  service  personnel  who  were  enrolled  In  BJC  or  BBC  during 
FY82.  No  distinction  was  Hade  between  those  who  completed  BJC  and  those  who  coMpleted  BBC. 
Fifty-seven  percent  of  the  sanple  were  Male,  S4t  were  white,  and  the  average  age  was  21.1  years. 
The  overall  acadeelc  failure  rate  In  the  sanple  was  18.91.  There  were  132  service 
representatives  froM  the  Arny,  and  48  from  the  Navy.  Since  the  Air  Force  and  the  Marine  Corps 
used  siHlIar  aptitude  coMposItes,  the  48  Individuals  from  these  services  were  coMblned  Into  one 
group  for  analytic  purposes. 


Data  Analysis 

Forward  hierarchical  Multiple  regression  (Cohen  A  Cohen,  1975)  was  performed  In  the  study. 
The  criterion  measure  was  the  final  grade  obtained  fro«  BJC  or  BBC.  Final  school  grades  are 
recorded  as  scores  ranging  between  50  and  100.  The  predictor  measures.  In  order  of  presentation 
In  the  regression  equations,  were  first,  the  ASVAB  composite  test  scores  from  the  Individuals' 
respective  service  and  second,  the  EOT  raw  score.  Composite  test  scores  preceded  the  EOT  scores 
In  the  hierarchical  regression  since  the  former  scores  would  be  available  before  the  latter 
scores. 


III.  RESULTS  AND  DISCUSSION 

Two  sets  of  multiple  regression  analyses  were  performed,  corresponding  to  two  different  ways 
of  treating  the  correlations.  First,  multiple  regression  analysis  utilized  correlations  that 
were  uncorrected  for  restriction  In  standard  deviations  due  to  selection.  The  results  of  these 
analyses  show  the  predictive  validities  that  are  obtained  directly  from  the  observed  data.  The 
saaiples  used  In  this  study  consisted  of  highly  qualified  Individuals  who  would  vary  less  In  their 
test  performance  than  the  general  population.  As  shown  In  Table  2,  the  restriction  In  talent  Is 
reflected  by  smaller  standard  deviations  In  the  composite  scores  for  each  sample.  A  second  set 
of  multiple  regression  analyses  utilized  correlations  that  were  corrected  for  these 
restrictions.  The  corrections  provided  estimates  of  correlations  based  on  the  full  range 
standard  deviations,  as  would  be  observed  In  an  unrestricted  sample. 

Table  3  summarizes  the  results  of  the  multiple  regression  analyses  when  all  four  services 
are  combined  Into  one  sample.  EOT  significantly  Increased  the  overall  predictive  validity  when 
either  the  CL  composite  or  the  ST  composite  was  used  as  the  Initial  predictive  composite.  When 
GEN  vas  used,  EDT  failed  to  significantly  Increase  overall  prediction.  Moreover,  the  predictive 
validity  of  GEN  alone  was  greater  than  the  overall  predictive  validities  of  the  CL  composite  and 
EDT  or  of  the  ST  composite  and  EOT.  Thus,  GEN  Is  the  most  valid  predictor  of  school  success. 


Table  2.  Population  and  Saiple  Standard  Oavlatlont  of  the 
Service  Co^>os1tes 


Subtest 

gen* 

Composite 

CL® 

ST' 

Population^ 

18.527 

16.017 

35.302 

Full  Sample 

8. 815 

12.749 

17.992 

Air  Force  and  Marine  Corps 

5.353 

12.974 

18.457 

Navy 

11.125 

8.692 

23.350 

Army 

8.412 

13.475 

14.319 

*GEN  •  General  composite  (AR  +  WK 
^CL  •  Clerical  composite  (WK  +  PC 
'ST  ■  Skilled  Technical  composite 

+  PC). 

+  NO  +  CS). 
(WK  +  PC  + 

GS  MK  + 

MC). 

‘^Standard  deviations  are  based  on  the  1980  Profile  of  American 
Youth  (Department  of  Defense,  1982). 


Table  3.  Prediction  of  Final  School  Grade  from  the 
Service  Composites  and  the  English  Diagnostic  Test  (EOT) 
for  the  Full  Sample  (N  ■  226) 


Predictors 

r2 

R^-change 

r2 

R^-change 

GEN* 

.282 

.634 

GEN.  EOT 

.293 

.011 

.640 

.006 

CL** 

.129 

.380 

CL,  EOT 

.186 

.057* 

.421 

.041* 

ST' 

.123 

.352 

ST,  EOT 

.157 

.034* 

.377 

.025* 

*GEH  ■  General  composite  (AR  «•  WK  +  PC). 

**01  «  Clerical  composite  (WK  +  PC  ♦  NO  +  CS). 

'ST  •  Skilled  Technical  composite  (WK  ♦  PC  +  GS  +  MK  r  MC). 
*r2  change  Is  significant  at  p  <  .01, 


The  data  were  also  analyzed  by  service  membership.  The  purpose  here  was  to  Investigate 
whether  the  results  of  the  full  sample  were  applicable  for  specific  service  groups. 

Table  4  summarizes  the  results  of  the  multiple  regression  analyses  for  a  sample  consisting 
of  Air  Force  and  Marine  Corps  service  personnel.  The  two  service  groups  were  combined  since  both 
services  use  similar  selection  levels  on  the  GEN  composite.  Inspection  of  the  uncorre-;ted  R^ 
values  reveals  poor  predictive  validities  for  all  service  composites,  alone  and  In  combination 
with  EOT,  When  the  correlations  are  corrected  for  restriction  In  range,  however,  the  corrected 
r2  values  supported  the  previous  finding  that  GEN  has  greater  predictive  validity  than  either 
combination  of  CL  and  COT  or  ST  and  EOT.  EOT  also  failed  to  Increase  overall  predictive  validity 
over  any  of  the  service  composites. 


Table  ♦.  Prediction  of  Final  School  Grade  froe  the 
Service  Coaposltes  and  the  English  Diagnostic  Test  (EOT) 
for  the  Coablned  Air  Force  and  Marine  Corps  Saaple  (N  •  46) 


Predictors 

R^  R^-change 

r2 

R^-change 

GEN* 

.081 

.515 

GEN,  EOT 

.090  .009 

.520 

.005 

cl'’ 

.097 

.303 

CL,  EOT 

.098  .001 

.303 

.000 

STC 

.026 

.068 

ST,  EOT 

.031  .005 

.093 

.005 

*GEN  •  General 
''CL  -  Clerical 
‘'ST  •  Skilled 

composite  (AR  +  WK  +  PC), 
composite  (UK  +  PC  +  NO  +  CS). 
Technical  composite  (UK  +  PC  +  GS 

+  HK  + 

MC). 

The  results  of  the  Multiple  regression  analyses  for  the  Navy  savple  are  sumMarIzed  In  Table 
5.  As  shown  earlier,  EOT  significantly  Increased  the  overall  predictive  validity  when  comtlned 
with  CL  or  ST.  This  result  also  holds  for  GEN,  although  when  corrected  values  are  used,  the 
Increase  In  R^  values  due  to  EOT  was  noninal.  The  second  previous  finding  was  also 
replicated;  GEN  alone  has  greater  predictive  validity  than  the  combined  predictive  validity  of 
either  CL  and  EOT  or  ST  and  EOT,  The  result  held  for  both  the  uncorrected  and  corrected  R^ 
values. 


Table  S.  Prediction  of  Final  School  Grade  froM  the 
Service  Composites  and  the  English  Diagnostic  Test  (EOT) 
for  the  Navy  Sample  (N  <  46) 


Uncorreeted  R^ 

Corrected  R^ 

Predictors 

r2 

R^-change 

R^ 

R^*change 

GEN* 

.495 

.731 

GEh.  EOT 

.595 

.100* 

.784 

.053* 

cl" 

.146 

.605 

:L,  EOT 

.415 

.269* 

.729 

.124* 

ST" 

.202 

.367 

Sf.  EOT 

.459 

.257* 

.571 

.204* 

*GEN  •  General 

composite  (AR 

♦  WK  +  PC). 

"CL  =■  Clerical 

composite  (UK 

+  PC  ♦  MO  +  CS). 

"ST  *  Skilled 

Technical  composite  (WK  ♦  PC  +  (iS 

♦  MK  + 

HC). 

*R^  change  Is 

signi  .ant  at 

p  <  .01. 

Table  6  summarizes  the  results  of  the  multiple  regression  analyses  for  the  Army  sample. 
These  results  also  agree  with  previous  findings.  Specifically,  GEN  alone  has  high  predictive 
validity,  and  EOT  contributes  an  Insignificant  Increase  In  explained  variance  beyond  that 
attributed  to  GEN. 


Table  6.  Prediction  of  Final  School  Grade  fro*  the 
Service  Conposltes  and  the  English  Diagnostic  Test  (EDT) 
for  the  Army  Saeple  (H  ■  J32) 


Predictors 

11  1  i-ri  1  T  ri  ■  ■  .-1  1  1 

Uncorrected  R‘ 

- - - - - -  1 ^  - 

Corrected  R^ 

R^-change 

r2 

R^-change 

GEN» 

.235 

,598 

GEN,  EOT 

.246 

.011 

.604 

.006 

CL*^ 

.183 

.455 

Cl.  EDT 

.219 

.036* 

.479 

.024* 

ST<^ 

.137 

.491 

ST.  EDT 

.160 

.023 

.504 

.013 

*GEN  •  General  coMposIte  (AR  +  WK  +  PC). 

**01  •  Clerical  coeiposlte  (WK  +  PC  +  NO  +  CS). 

=  ST  -  Skilled  Technical  composite  (WK  +  PC  +  GS  +  MK  -t  MC). 

*r2  change  Is  significant  at  <  .05. 

The  results  from  the  multiple  regression  analyses  for  each  of  the  services  and  for  the  full 
sample  may  be  summarized  as  follows: 

1.  EDT  significantly  increases  predictive  validity  when  combined  with  either  the  CL  or  ST 
composites. 

2.  GEN  composite  alone  has  greater  predictive  validity  than  either  of  the  preceding 
combinations  of  measures. 

3.  EDT  and  GEN  together  do  not  have  significantly  higher  predictive  validity  than  GEN  alone. 


if.  RECONNENDATiONS 

The  following  observations  and  recommendations  are  offered: 

1.  Use  the  GEN  composite  for  qualifying  all  enlisted  service  personnel  for  entry  Into  BJC 
and  BBC.  The  GEN  composite  has  greater  validity  tnan  either  CL  or  ST.  If  the  GEN  composite  is 
used,  EOT  does  not  increase  the  validity  for  predicting  school  success.  Adopting  the  GEN 
composite  has  the  further  advantage  of  using  available  Information  and  does  not  require  the  added 
cost  (time,  material,  and  manpower)  of  administering  the  LDT. 

A  cutoff  score  corresponding  to  the  65th  percentile  on  the  GEN  Is  recommended.  A  GEN 
score  below  the  65th  percentile  Is  not  recommended  because  the  admittance  of  a  greater  number  of 
unqualified  personnel  Increases  the  need  for  remedial  training;  moreover,  the  academic  failure 
rate  may  increase.  Figure  1  shows  the  academic  failure  rate  of  school  entrants  for  different 
cutoff  levels  of  the  GEN  oercentlle.  Fcr  example.  In  this  study,  tne  academic  failure  rate  of 
personnel  with  GEN  65  percentile  or  higher  was  0.0*.  compared  to  the  overall  academic  failure  rate 
of  16. 9t.'  Additionally,  no  significant  decreases  in  attrition  are  obtained  until  a  cutoff 
score  of  85  is  applied.  A  GEN  scene  above  the  65th  percentile  is  not  recommended  because  the 
percent  of  Ineligible  personnel  below  a  pariic.lar  GEN  cutoff  score  Increases  as  the  GEN  cutoff 
score  increases.  The  Increase  becomes  most  apparent  at  GEN  85.  The  number  of  personnel  who 
would  have  to  be  considered  in  an  applicant  pool  to  vield  100  graduates  is  shown  in  Figure  2  for 
Increasing  levels  of  selectivity.  Although  attrition  rates  would  decrease  as  successively  higher 
cutoffs  were  established,  this  would  require  a  dramatic  Increase  in  the  applicant  pool, 
especially  beyond  GEN  80.  On  this  basis,  the  GEN  65  selection  score  is  preferred. 


^An  Individual  who  obtains  a  grade  of  less  than  70  points  is  considered  to  ire  an  academic 

failure. 


GEN  Percentile  Cutoff  Score 


2.  If  a  cutoff  score  corresponding  to  the  65th  percentile  leads  to  Insufficient  course 
enrollaient,  a  lower  cutoff  score  should  be  used  to  allow  a  greater  nuwber  of  personnel  Into  the 
prograMS.  This  alternative,  however,  will  also  adult  a  greater  number  of  less  qualified 
Individuals.  As  a  consequence,  an  Increased  need  for  remedial  reading  training  and  Increased 
academic  failure  rate  may  result.  Reviewing  the  data  In  Figures  I  and  2  will  help  In  reaching  an 
operational  decision. 
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